Aggregation via Empirical Risk Minimization
نویسندگان
چکیده
Given a finite set F of estimators, the problem of aggregation is to construct a new estimator whose risk is as close as possible to the risk of the best estimator in F . It was conjectured that empirical minimization performed in the convex hull of F is an optimal aggregation method, but we show that this conjecture is false. Despite that, we prove that empirical minimization in the convex hull of a well chosen, empirically determined subset of F is an optimal aggregation method.
منابع مشابه
Aggregation versus Empirical Risk Minimization
Abstract Given a finite set F of estimators, the problem of aggregation is to construct a new estimator that has a risk as close as possible to the risk of the best estimator in F . It was conjectured that empirical minimization performed in the convex hull of F is an optimal aggregation method, but we show that this conjecture is false. Despite that, we prove that empirical minimization in the...
متن کاملEmpirical risk minimization is optimal for the convex aggregation problem
Let F be a finite model of cardinality M and denote by conv(F ) its convex hull. The problem of convex aggregation is to construct a procedure having a risk as close as possible to the minimal risk over conv(F ). Consider the bounded regression model with respect to the squared risk denoted by R(·). If f̂ ERM-C n denotes the empirical risk minimization procedure over conv(F ), then we prove that...
متن کاملSuboptimality of Penalized Empirical Risk Minimization in Classification
Let F be a set of M classification procedures with values in [−1, 1]. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in F . This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various types of convexity, we prove that optimal rates of aggregation can be either ((logM)/n)...
متن کاملPerformance of empirical risk minimization in linear aggregation
Let (X ,μ) be a probability space, set X to be distributed according to μ and put Y to be an unknown target random variable. In the usual setup in learning theory, one observes N independent couples (Xi, Yi)Ni=1 in X × R, distributed according to the joint distribution of X and Y . The goal is to construct a real-valued function f which is a good guess/prediction of Y . A standard way of measur...
متن کامل[hal-00736203, v1] Empirical risk minimization is optimal for the convex aggregation problem
Let F be a finite model of cardinality M and denote by conv(F ) its convex hull. The problem of convex aggregation is to construct a procedure having a risk as close as possible to the minimal risk over conv(F ). Consider the bounded regression model with respect to the squared risk denoted by R(·). If f̂ n denotes the empirical risk minimization procedure over conv(F ) then we prove that for an...
متن کامل